Clinical prediction modelling in oral health: A review of study quality and empirical examples of model development

Background Substantial efforts have been made to improve the reproducibility and reliability of scientific findings in health research. These efforts include the development of guidelines for the design, conduct and reporting of preclinical studies (ARRIVE), clinical trials (ROBINS-I, CONSORT), observational studies (STROBE), and systematic reviews and meta-analyses (PRISMA). In recent years, the use of prediction modelling has increased in the health sciences. Clinical prediction models use information at the individual patient level to estimate the probability of a health outcome(s). Such models offer the potential to assist in clinical decision-making and to improve medical care. Guidelines such as PROBAST (Prediction model Risk Of Bias Assessment Tool) have been recently published to further inform the conduct of prediction modelling studies. Related guidelines for the reporting of these studies, such as TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis) instrument, have also been developed. Since the early 2000s, oral health prediction models have been used to predict the risk of various types of oral conditions, including dental caries, periodontal diseases and oral cancers. However, there is a lack of information on the methodological quality and reporting transparency of the published oral health prediction modelling studies. As a consequence, and due to the unknown quality and reliability of these studies, it remains unclear to what extent it is possible to generalise their findings and to replicate their derived models. Moreover, there remains a need to demonstrate the conduct of prediction modelling studies in oral health field following the contemporary guidelines. This doctoral project addresses these issues using two systematic reviews and two empirical analyses. This thesis is the first comprehensive and systematic project reviewing the study quality and demonstrating the use of registry data and longitudinal cohorts to develop clinical prediction models in oral health. Aims • To identify and examine the quality of existing prediction modelling studies in the major fields of oral health.• To demonstrate the conduct and reporting of a prediction modelling study following current guidelines, incorporating machine learning algorithms and accounting for multiple sources of biases. Methods As one of the most prevalent oral conditions, chronic periodontitis was chosen as the exemplar pathology for the first part of this thesis. A systematic review was conducted to investigate the existing prediction models for the incidence and progression of this condition. Based upon this initial overview, a more comprehensive critical review was conducted to assess the methodological quality and completeness of reporting for prediction modelling studies in the field of oral health. The risk of bias in the existing literature was assessed using the PROBAST criteria, and the quality of study reporting was measured in accordance with the TRIPOD guidelines. Following these two reviews, this research project demonstrated the conduct and reporting of a clinical prediction modelling study using two empirical examples. Two types of analyses that are commonly used for two different types of outcome data were adopted: survival analysis for censored outcomes and logistic regression analysis for binary outcomes. Models were developed to 1) predict the three- and five-year disease-specific survival of patients with oral and pharyngeal cancers, based on 21,154 cases collected by a large cancer registry program in the US, the Surveillance, Epidemiology and End Results (SEER) program, and 2) to predict the occurrence of acute and persistent pain following root canal treatment, based on the electronic dental records of 708 adult patients collected by the National Practice-Based Research Network. In these two case studies, all prediction models were developed in five steps: (i) framing the research question; (ii) data acquisition and pre-processing; (iii) model generation; (iv) model validation and performance evaluation; and (v) model presentation and reporting. In accordance with the PROBAST recommendations, the risk of bias during the modelling process was reduced in the following aspects: • In the first case study, three types of biases were taken into account: (i) bias due to missing data was reduced by adopting compatible methods to conduct imputation; (ii) bias due to unmeasured predictors was tested by sensitivity analysis; and (iii) bias due to the initial choice of modelling approach was addressed by comparing tree-based machine learning algorithms (survival tree, random survival forest and conditional inference forest) with the traditional statistical model (Cox regression). • In the second case study, the following strategies were employed: (i) missing data were addressed by multiple imputation with missing indicator methods; (ii) a multilevel logistic regression approach was adopted for model development in order to fit Table of Contents xi the hierarchical structure of the data; (iii) model complexity was reduced using the Least Absolute Shrinkage and Selection Operator (LASSO) for predictor selection; and (iv) the models’ predictive performance was evaluated comprehensively by using the Area Under the Precision Recall Curve (AUPRC) in addition to the Area Under the Receiver Operating Characteristic curve (AUROC); (v) finally, and most importantly, given the existing criticism in the research community concerning the gender-based and racial bias in risk prediction models, we compared the models’ predictive performance built with different sets of predictors (including a clinical set, a sociodemographic set and a combination of both, the ‘general’ set). Results The first and second review studies indicated that, in the field of oral health, the popularity of multivariable prediction models has increased in recent years. Bias and variance are two components of the uncertainty (e.g., the mean squared error) in model estimation. However, the majority of the existing studies did not account for various sources of bias, such as measurement error and inappropriate handling of missing data. Moreover, non-transparent reporting and lack of reproducibility of the models were also identified in the existing oral health prediction modelling studies. These findings provided motivation to conduct two case studies aimed at demonstrating adherence to the contemporary guidelines and to best practice. In the third study, comparable predictive capabilities between Cox regression and the non-parametric tree-based machine learning algorithms were observed for predicting the survival of patients with oral and pharyngeal cancers. For example, the C-index for a Cox model and a random survival forest in predicting three-year survival were 0.82 and 0.84, respectively. A novelty of this study was the development of an online calculator designed to provide an open and transparent estimation of patients’ survival probability for up to five years after diagnosis. This calculator has clinical translational potential and could aid in patient stratification and treatment planning, at least in the context of ongoing research. In addition, the transparent reporting of this study was achieved by following the TRIPOD checklist and sharing all data and codes. In the fourth study, LASSO regression suggested that pre-treatment clinical factors were important in the development of one-week and six-month postoperative pain following root canal treatment. Among all the developed multilevel logistic models, models with a clinical set of predictors yielded similar predictive performance to models with a general set of predictors, while the models with sociodemographic predictors showed the weakest predictive ability. For example, for predicting one-week postoperative pain, the AUROC for models with clinical, sociodemographic and general predictors were 0.82, 0.68 and 0,84, respectively, and the AUPRC were 0.66, 0.40 and 0.72, respectively. Conclusion The significance of this research project is twofold. First, prediction models have been developed for potential clinical use in the context of various oral conditions. Second, this research represents the first attempt to standardise the conduct of this type of studies in oral health research. This thesis presents three conclusions: 1) Adherence to contemporary best practice guidelines such as PROBAST and TRIPOD is limited in the field of oral health research. In response, this PhD project disseminates these guidelines and leverages their advantages to develop effective prediction models for use in dentistry and oral health. 2) Use of appropriate procedures, accounting for and adapting to multiple sources of bias in model development, produces predictive tools of increased reliability and accuracy that hold the potential to be implemented in clinical practice. Therefore, for future prediction modelling research, it is important that data analysts work towards eliminating bias, regardless of the areas in which the models are employed. 3) Machine learning algorithms provide alternatives to traditional statistical models for clinical prediction purposes. Additionally, in the presence of clinical factors, sociodemographic characteristics contribute less to the improvement of models’ predictive performance or to providing cogent explanations of the variance in the models, regardless of the modelling approach. Therefore, it is timely to reconsider the use of sociodemographic characteristics in clinical prediction modelling research. It is suggested that this is a proportionate and evidence based strategy aimed at reducing biases in healthcare risk prediction that may be derived from gender and racial characteristics inherent in sociodemographic data sets.Thesis (Ph.D.) -- University of Adelaide, School of Public Health, 202

Travel fatigue and home ground advantage in South African Super 12 rugby teams

Objectives. Studies have shown the existence of a significant home ground advantage in a range of sports. The aim of this study was to determine whether home performances of the South African teams during the 1996 - 2005 seasons were different before touring to Australia and New Zealand, compared with the home matches played after the 4 - 5-week tour. The question was relevant because this competition places unusual demands on the players. For example, the duration of the tournament is 12 weeks, during which time the South African teams travel across 37.3 ± 2.5 time zones as they play 11 matches. Therefore, any home ground advantage may be negated by travel fatigue. Outcome measures. The mean points difference was calculated for home fixtures for four South African teams both prior to and following touring during the competitions from 1996 to 2005. Play&#8209;off matches were excluded from analysis. Performance was defined as a positive points difference (points difference = points ‘for' minus points ‘against'). Results. The first finding of the study was that a home ground advantage did indeed exist for all the teams during the tournament (points difference of 6.6 ± 17.4 (N = 664 matches) home vs. -6.8 ± 17.3 points away (N = 656 matches) (p < 0.05). There was no difference between the mean ‘home' points difference for all the South African rugby teams either before (1.9 points, N = 96 matches) or after (2.3 points, N = 107 matches) touring overseas in the Super 12 competition. South African Journal of Sports Medicine Vol. 19 (1) 2007: pp. 20-2

The effects of environmental inspection on air quality: Evidence from China

To address ecological and environmental issues, central environmental inspection (CEI) coordinated by the Chinese Ministry of Ecology and Environment has been implemented since 2016. This paper aims to comprehensively evaluate how and how much CEI affects air quality. The results of the difference-in-differences models show that CEI improved the air quality and reduced the concentrations of PM2.5, PM10, NO2, and SO2 by 8.8%, 8.1%, 7.9%, and 2.4%, respectively. Moreover, environmental effectiveness was strengthened over the course of four rounds of inspection. The mediating model results indicate that effectiveness was achieved through active public participation, administrative punishments from the central inspectors, and positive rectification actions from the local governments. The greatest improvement in air quality occurred during the on-site inspection period, after which the effects gradually weakened. A review inspection was carried out to supervise the rectification tasks. The adoption of review inspection made the effects on air quality improvement reappear, which verifies that CEI in China is not just a temporary campaign-style enforcement but a normalized and effective governance of air pollution

Do Facebook Activities Increase Sales?

Facebook is a one of the most popular social media platforms and its increasing adoption by business is leading to the shift of traditional marketing to social marketing (Nair 2011). This study investigates two related questions: 1) whether the use of Facebook impacts companies’ sales; 2) whether the increased Facebook activities leads to higher companies’ sales. We find that, on average, companies adopted Facebook have sales 0.1% higher than those not. We also find if a company increases its Facebook posts (interactions) by 1%, its annual sales will increase by roughly 0.06% (0.03%). Our study provides evidence that Facebook activities are significantly and positively associated with companies’ annual sales though their impacts are relatively small in terms of effect size. We also provide caveats to the interpretation of our results and discuss directions for future research

Money Talks: A Predictive Model on Crowdfunding Success Using Project Description

Existing research of crowdfunding mainly focuses on the basic properties of the project such as category and goal, the information content of the project, however, is barely studied. By introducing Elaboration Likelihood Model into crowdfunding context and using a large dataset obtained from Kickstarter, a popular crowdfunding platform, we study the influence of project descriptions in terms of argument quality and source credibility, and investigate their impacts on funding success. We find information disclosed in project descriptions is associated with funding success. We also examine the practical impacts of project description by using a predictive model. Results show that our model can predict with an accuracy rate of 73% (71% in F-measure), which represents an improvement of 15 percentage points over the baseline model and 4 percentage points over the mainstream model. Overall, our results provide insights to researchers, project owners and backers to better study and use crowdfunding platforms

Travel fatigue and home ground advantage in South African Super 12 rugby teams

Objectives. Studies have shown the existence of a significant home ground advantage in a range of sports. The aim of this study was to determine whether home performances of the South African teams during the 1996 - 2005 seasons were different before touring to Australia and New Zealand, compared with the home matches played after the 4 - 5-week tour. The question was relevant because this competition places unusual demands on the players. For example, the duration of the tournament is 12 weeks, during which time the South African teams travel across 37.3 ± 2.5 time zones as they play 11 matches. Therefore, any home ground advantage may be negated by travel fatigue. Outcome measures. The mean points difference was calculated for home fixtures for four South African teams both prior to and following touring during the competitions from 1996 to 2005. Play&#8209;off matches were excluded from analysis. Performance was defined as a positive points difference (points difference = points ‘for' minus points ‘against'). Results. The first finding of the study was that a home ground advantage did indeed exist for all the teams during the tournament (points difference of 6.6 ± 17.4 (N = 664 matches) home vs. -6.8 ± 17.3 points away (N = 656 matches) (p < 0.05). There was no difference between the mean ‘home' points difference for all the South African rugby teams either before (1.9 points, N = 96 matches) or after (2.3 points, N = 107 matches) touring overseas in the Super 12 competition. South African Journal of Sports Medicine Vol. 19 (1) 2007: pp. 20-2